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Near Optimum Error Correcting 
Coding And Decoding: Turbo-Codes 
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Ahmet— This paper presents -a new family or conTohOtonnl 
codes, nicknamed turbo-code*, boDt from a particular concate- 
nation of two recursive systematic codes, linked together by 
nonnniform interleaving. Decoding caQs on iterative processing 
In which each component decoder takes advantage of tbe work 
of me other at the previous step, wftb the aid of tbe original con* 
cept of extrinsic information. For snfGckndy large interleaving 
sixes, the correcting performance of turbo-codes, hrr estimated by 
jdmniati offi, appears to be close to tbe theoretical limit predicted 
by Shannon, 



L Introduction 

CONVOLUTIONAL error correcting or channel coding 
hre become widespread in the design of digital transmis* 
sion systems. One major reason for mis is the possibility of 
achieving real-time decoding without noticeable information 
Losses thanks to the well-known -soft-input Viterbi algorithm 
[1]. Moreover, the same decoder may serve for various coding 
rates by means of puncturing [2], allowing tbe same silicon 
product to be used in different applications. TWo kinds of 
coavolutional codes are of practical interest: nonsystematic 
convolurional (NSC) and recursive systematic convolurional 
(RSC) codes. Ibough RSC codes have the same free distance 
d/ as NSC codes and exhibit better performance at low signal 
to noise ratios (SNR's) and/or when ptmcftured, only NSC 
sodes have actually been considered for channel coding, except 
a Trelliscoded modulation <JCM) [3]. Section II presents the 
principle and the rjerformance of RSC codes, which are at me 
"OOt of the study expounded in this article, 
i For a given rate, the error^xirrectiiig power of convolurional 
odes, measured as the coding gain at a certain binary error rate 
BER) in comparison with the uncoded traiisinission, grows 
nore or less lineariy with code memory v. Fig. 1 (from [4D 
hows the achievable coding gains for different rates* and 
orresponding bandwidth expansion rates, by using classical 
BC codes with v = 2, 4, 6 and 8, fox a BER of 10" tt . For 
^stunce, with R — 1/2, each unit added to v adds about 0.5 
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Hg. I. Ceding font at BER equal to 10~", achievable with NSC code* 
(thne-bit quantization from [4]. and maximum possible gains for 172, 2/3, 
3/4- md 4/5 tates in a Gmis&inn channel, with quaxemary phase ihift keying 
(QPSK) modulation 



dB more to the coding gain* up to v = 6; for u = 8, the 
additional gain is lower. Unfortunately, the complexity of the 
decoder is not a linear function of u and it grows exponentially 
as v - 2*. Factor 2 represents the number of states processed 
by the decoder and the multiplying factor v accounts for the 
complexity of the memory part (metrics and survivor memory)* 
Other technical limitations like the interconnection constraint 
in the silken decoder lay-out* make die value of six a practical 
upper limit f or v for most applications, especially' for high 
data rates. 

In order to obtain high coding gains with moderate decoding 
complexity,, concatenation has proved to be an attractive 
scheme. Classically, concatenation has consisted in cascading 
a block code (the oarer code, typically a Reed-Solomon code) 
and a convolutional code (the inner code) in a serial structure. 
Another concatenated code, which has-been given the familiar 
name of turbo-code, with an original parallel organization of 
two RSC elementary codes, is described in Section TJL Some 
comments about the distance properties of this composite 
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code, arc propounded. When decoded by an iterative process, 
tuibo-codes offer near optimum performance. The way to 
achieve this decoding with the Maximum A Posteriori (MAP) 
algorithm is detailed in Sections IV, V. VI, and some basic 
results ana given in Section VtL 

IL Recursive Systematic Convolutional Codes 

A introduction 

Consider a binary rate R — 1/2 convolutional encoder with 
constraint length K and memory v — K - 1. The input to 
the encoder at time Ji: is a bit d k and the corresponding binary 
couple (Xk>Yk) is equal to 



(la) 



(lb) 



where Git{gu) 9 <h'*{&i} m ^ two 61100(161 generators, 
expressed in octal ftm It is well known that the BER 
of a classical NSC code is lower than that of a classical 
nonrecursive systematic convolutional code with the same 
memory v at large SNR's, since its free distance is smaller 
man that of a NSC code \S\ At low SNR's, it is in general the 
other way round. The RSC code; presented below, combines 
the properties of NSC and systematic codes- In particular, it 
can be better than the equivalent NSC code, at any SNR. for 
code rates larger man 2/3. 

A binary rate RSC code is obtained from a NSC code by 
using a feedback loop and setting one of the two outputs Xk 
or Yk equal to the input bit dk. The shift register (memory) 
input is no longer the bit dk but is a new binary variable a*. 
TTXt - d k (respectively, Y k = <fc), the output Y h (tesp. X k ) 
is defined by (lb) [respectively, (la)) by substituting dk for 
a* and variable a* is recursively calculated as 



a* ~ dk + 7* a *-» 



(2) 



iml 



where rti is respectively equal to gu if X b = d k and to 921 if 
Yk = dk* Bquation (2) can be rewritten as 



(3) 



Taking into account X k = d k or Y k =: d*. the RSC encoder 
output I?* — {Xk,Yk) has exactly the same expression as the 
NSC encoder outputs if sxd = £20 ~ 1 and by substituting d k 
for <J fc in (la) or (lb). 

Two RSC encoders with memory v = 2 and rate 12 = 1/2, 
obtained from a NSC encoder defined by generators G\ = 
7, £2 = 5, are depicted" in Fig> 2. 

Generally, we assume that the input bit d k takes values zero 
or one with the same probability.' prom (2), we can show that 
variable a k exhibits the same statistical property 



Pr{a* = w 



(4) 



42 



x k 



<b) 

Fig> 2. Two associated R*cumv« vyBtcmmfc oonwoJutioiwI (RSC) encoders 
with memory u as 2, rate R = 1/2 and gpner&Jort Ci = 7,<?a = 3. 



where e is equal to 



£ = ^ 7^1 = 0, 1. 



(5) 



Thus, the transition state probabilities ic{8 k — m/S k -i — 
m')» where 5* - in and £ fc _i = m' are, respectively* the 
encoder state at time k and at rime (k - 1), are identical for 
the equivalent RSC and NSC codes; moreover these two codes 
have the same free distance d/. However, for a same input 
sequence me two output sequences {Xk} and {%} are 
different for RSC and NSC codes. 

When pwttturing is considered* some output bits X k or Yk 
are deleted according to a chosen perforation pattern defined 
by a matrix JP, For instance, starting from a rate R — 1/2 
code, the matrix P of rate 2/3 punctured code can be equal to 



(6) 



For the punctured RSC code, bit d* must be emitted at each 
time h. This is obviously done if all the elements belonging 
to the first or second row of matrix P are equal to one. When 
the best perforation pattern for a punctured NSC code is such 
that matrix P has null elements in the first and the second 
rows, the punctured recursive cc>nvoltirionfll code is no longer 
systematic. In order to use the same matrix P for both RSC 
and NSC codes, the RSC encoder is now defined by (3). (7). 
and (8) 



Xk =dk 



(7) 
(8) 
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ere coefficients 7 y [see (3)J and A; are, respectively, equal 
7il and Qk when element p^; I < j < n of matrix P i$ 
ial to one and to fa and g u when pi,- is equal to zero. 

Recursive Systematic Code Performance 
n order to compare the performance of RSC and NSC 
ea, we determined their weight spectrum and their BER. 

weight spectrum of a code is made up of two sets 
coefficients a(d) and W{d) obtained from two series 
anskras related to the code transfer function T(P, N) [6J 
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(9) 



CIO). 



ne d/ is the free distance of the code, a{d) is the number 
atbs at Hamming distance of from the "nulT path and 
!) is we total Hamming weight of input sequences {4} 

to generate all paths at distance d from the "nulT path, 
-social, it is not easy to calculate the transfer function 

punctured code, that is why the first coefficients a(d) 
W(d) are directly obtained from the trellis by using an 
ithm derived from [7J; From coefficients W(d), a tight 
r bound of error probability can be calculated for large 
'& 16) 




memoryless Gaussian channel with binary modulation 
. QPSK), probability P(d) is equal to 



4 5 6 7 8 
EWNo (dB) 

Pig. 3, P, of punctured RSC and NSC codes for diffeicni values of rue Jl 
and memoiy v as 6, generators <? t c: 133«Cft it 171, \ 

is approximately 0.8 dB at R = 2/3, whereas at R = 3/4, 
it reaches 1.75 dB. 



J><d) 



£jb/Ar 0 is the energy per information bit to noise powi 
al density ratio and i? is me code rate, 

a large number of RSC codes have been investigatec 
sir performance was compared to that of NSC codes, 
rt of weight spectrum and of BER. Coefficients a{d) 
3 same for RSC and NSC codes but the coefficients 
:(<*)} of RSC codes have a tendency to increase more 

in function of rf than the coefficients {Wu&c(d)} of 
odes, whatever the rate R and whatever the memory 
s, at low SNR's, the BER of the RSC code is always 

* man the BER of the equivalent NSC code, 
isoeral, for rates R < 2/3, the first coefficients 

WrscW/ 4, l)) are larger than those of NSC 
therefore, at large SNRVtbe performance of NSC 

* a little better than that of RSC codes. When the 
larger than 2/3, it is easy to find RSC codes whose 
lance is better man that of NSC codes at any SNR, 
der to illustrate the performance of RSC codes* the 
RSC and NSC codes are plotted in Fig. 3 for different 
of R and for an encoder with memory u = 6 and 
ws 133, 171. For instance, at P c ea Ur 1 , the coding 
h this; RSC code, relative to the equivalent NSC code. 




EL Parallel Concatenation with Non 
Uniform Interleaving: Turoo-Coob 



A Construction of the Code 

The use of systematic codes enables the construction of 
a concatenated encoder in the form given in Pig. 4, called 
parallel concatenation. The data flow (d h at time k) goes 
directly to a first elementary RSC encoder <7 2 and after 
mterieaving, it reeds (d* at time k) a second elementary 
RSC encoder ft. These two encoders are not necessarily 
identical. Data d k is systematically transmitted as symbol 
X k and redundancies Y lk and Y 2h produced by Ci and C 2 
may be completely tranminad for flrt ^ j ^ encodm y w 
punctured for higher mtes./ inetwo elementary coding rates 
tti and R* associated with Ci and C 2i after puncturing, may 
be different, but for the best decoding performance, they will 
M&ifyRi < Ri- The global rate R of me composite code, R x 
and R 2 are linked by 



1 - i _L 

R~ Ri* R** 1 ' 



(13) 



Unlike me classical (serial) concatenation, parallel concate- 
nation enables the elementary encoders, and therefore the 
associated elementary decoders, to run with the same clock. 
This point constitutes an important simplification for the 
design of the associated circuits, m a concatenated scheme. 
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Basic Rntxheaaxtor (rate l/3> 




BSC code C2 



A. Distance Properties 

prtflgiHw for instance elementary codes <7i and G 3 with 
mernory v = 4 and encoder polynomials Gi = 23, Gj = 35. 
Redundancy Yk is one time every second time, either Yih or 
Y 2hl Hiefl the global rate is J* = 1/2 with i2i = = 
2/3. The code being linear, toe distance properties will be 
considered relatively to the "all zero** or "null" sequence. Both 
encoders Ci,C 2 and the interleaver are initialized to the "all 
zero" state and a. sequence {d k } containing w "l"s. is fed to 
the turbo-encoder, w is called the J>ip"f weight. 

$ome definitions: let us call a /wfc Codeword (FQ of 
an elementary RSC encoder an output sequence with a finite 
distance from the "all zero" sequence (i,e„ a limited number 
of 'Ts in output sequences {Xt} and {Yk}). Because of 
lecuisrvity, only some input sequences, fitting with the linear 
feedback register (LFR) generation of Y k , give PC's. These 
particular input sequences axe named FC (input) patterns. Let 
us also call a global FC an output sequence of the turbo-code, 
with a finite distance from the "all zero" sequence (Le., a 
limited number of *T*S in output sequences {X*}, {Yxb} and 
{Yat}). The distance d 9 (w) of an elementary PC (q - 1 for 
C u q 2 for Ca), associated with an input sequence {tZ*} 
with weight uij is the sum of the two contributions of {Xk} 
and {Y qk } 

<*,(*>) - djc v (t<0 + <*r ff (*0« (14) 
Since the codes are systematic, djr g (w) = if 

<i,(w) = *> + oV,(uO. (15) 
The distance d(w) of a global FC is given litewise by 

d(ui) rs w + dyi(ui) + dy2(tt0* (1© 
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at T' is 15 or a multiple of 15, since the LFR associated 
with the redundancy generation, with polynomial <r — 23> is 
a maximum length LFR- If the delay is not a multiple of 15. 
then the associated sequence will contain D"; and "l"s 
indefinitely. i 

Global FC's with Input Weight u> = 2 

Global PCs with w = 2 have to be PCs with mput weight 
to 2 for each of the constituent codes. Then the two data at 
w l* in the interleaving memory must be located at places which 
are distant by a mnlnple of 1 5, when both writing and reading. 
For lack of an extensive analysis of the possible patterns which 
satisfy this double constraint, let us consider only the case 
where the two Ts in {d k } are time^separated by the lowest 
value (L e. 15). It is also a PC pattern for code C% if the two 
data at *T" are memorized on a single line, when wt ting in the 
interleaving memory, and thus the span between th > two "l^s 
is 15 • Af , when reading, Rom (16), the <u5tance|associated 
with this input pattern is approximately equal to 



d(2) « 2 + mm(d Yl (2)} + INT((15 -M + l>/4) 



^2 + 4 + lNT((15'Af +l)/4) 



(17) 



I) Uniform Interleaving; Consider a uniform block inter- 
leaving using an Af > Af square matrix with M large enough 
>32), and generally a power of 2. Data are written 
lipcwse and read cohiznnwise. As explained above, the matrix 
is filled with "0"s except for some Ts and we are now going 
to state some of the possible patterns of "l"s leading to global 
FC* s add evaluate their associated distances. Beforehand, note 
mat, for each elementary code, the minimal value for input 
weight vj is two, because of their recursive structure. For the 
particular case of u* — 2, the delay between the two data 



where INT( ) stands for integer part of (<)• 'Ehe first term 
represents input weight w t the second term the distance added 
by redundancy Y\ of Ci (rate 2/3), the third term the distance 
added by redundancy Y 2 < The latter is given assunnng that . 
for this second 2/3 rate code, the (15 - M + l}/2 *a symbols 
are at "1", one out of two times statistically, which explains 
the additional division by 2. With Af = 64 for instance, the 
distance is d(2) « 246. This value is for a particular case of a 
pattern of two "l**s but we imagine it is at*alistic example for 
all PC's with input weight 2. If the two Ts are not located 
on a single line when writing in the interleaving matrix, and 
if Af is larger than 30, the span between the two "l"s, when 
reading from the mterleaYing matrix, is higher than 15 -Af and 
the last term in (17) is increased. 

Global FC's with in = 3 

Global FC* s with input weight w = 3 have to be elementary 
FC's with input weight w =r 3 for each of the .constituent 
codes. The inventory of the patterns with three f*l**s which 
satisfy this drabk coristraiirt 

to give the sHghtest example. Nevertheless, we can consider 
that associated distances are similar to the case of input weight 
two codewords, because the PC f or Ca is still several times 
Af long. 

Global FC 's with w = 4 

Here is the first pattern of a global FC which can be viewed 
as the separate combination of two minimal (input "weight 
w = 2) elementary FC patterns both for & and <V When 
writing in the interleaving memory, the four data at "1" are 
located at the four comers of a square or a rectangle, the 
sides of which have lengths equal to 15 or a multiple of 45 
[Fig* 5(a)]. The imnimum distance is given by a square pattern, 
with side length equal to 15, ce*iespc*idiiig to minimum values 
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